Gauge-field fiuctuations in the 3D topological Mott insulator 
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We study the low-energy properties of three-dimensional (3D) topological Mott insulators which 
can be viewed as strong topological insulators of spinous interacting with a three-dimensional gauge 
field. The low-energy behavior of such systems is dominated by the two-dimensional (2D) gapless 
surface spinous coupled to the bulk gauge field. We find that a dimensional crossover from 3D 
to 2D in the gauge field fluctuations may occur as the systems thickness and/or temperature is 
varied. In the thin sample limit, the gauge field fluctuations effectively become 2D and the problem 
becomes analogous to the standard 2D spinon-gauge field theory. In the 3D limit, the bulk gauge 
field fluctuations lead to a low-energy theory for the coupled system that is more controlled than 
for the pure 2D case. We discuss various experimental signatures such as the heat capacity scaling 
as Tlnl/T as well as modified Ruderman-Kittel-Kasuya-Yoshida interactions on the surface. 



I. INTRODUCTION 

Topological insulators of non-interacting fermions are 
characterized by topological invariants of the band 
structure.EHi^! While the bulk spectrum is gapped, the 
boundary states of such systems carry the characteristic 
informations about the topology of the bulk band struc- 
ture. This important observation has been instrumen- 
tal to the experimental identifications of the 2D and 3D 
topological insulators .'i^"IS In particular, 3D strong topo- 
logical insulators are characterized by the presence of an 
odd number of Dirac-cone spectra.EISSIin! These "helical 
liquids" or Dirac fermions with a finite chemical poten- 
tial provide low energy excitations in otherwise gapped 
systems. The presence of such excitations and their he- 
lical nature, the "locking" of momenta and spins, were 
beautifully confirmed in recent experiments! "^^ 1 

It is natural to ask what would be the analogous topo- 
logical phases in interacting many-body systems. A well- 
known example of such correspondence would be the 
case of integer and fractional quantum Hall states al- 
beit they are time-reversal symmetry breaking phases 
in contrast to the topological insulator .1^21 There have 
been various theoretical proposals for "interacting" topo- 
logical insulators.^" ■^''^ Some of these theories use the 
so-called "parton construction". Here the electron is 
"split" into partons carrying fractional quantum num- 
bers, where the individual partons are in topological 
insulator phases.'23n23 On the other hand, in the con- 
text of Mott insulators, the construction via the slave- 
rotor formulatiorPSHSS lias been proposed, where the elec- 
tron Hilbert space is written as a product of the spin- 
1/2 neutral spinon states and rotors representing charge 
coherence .'SSHIIIMEIl Because of the constraint coming 
from the original electron Hilbert space, these excita- 
tions are strongly coupled to a U(l) gauge field. The 
slave-rotor field theory can, for example, naturally de- 
scribes an insulator-metal transition via the condensa- 
tion of the rotor degrees of freedom. Moreover, the re- 
sulting Mott insulators are spin liquids where the spinons 
are natural low energy excitations.'^SHSIl When one starts 
from a strong topological insulator of weakly interacting 



electrons, instead of a metal, the increased interaction 
may allow the transition to a Mott insulator state by 
gapping out the rotor degrees of freedom (or destroy- 
ing charge coherence). The resulting state is again a 
spin liquid, but the spinons in this case would inherit 
the topological band structure of the original electrons. 
These "topological" spin liquid states are called topolog- 
ical Mott insulators.—"— Recently it has been proposed 
that certain pyrochlore iridates materials as well as other 
transition m etal o xides may be good candidates for such 
novel phases .l^il^SlEi] Qne advantage of this theory is that 
it naturally connects the topological insulators of weakly 
interacting electrons to topological Mott insulators where 
electrons are no longer well-defined excitations because 
of strong interactions. 

In analogy to the topological insulators, the boundary 
state should carry the characteristic information about 
the topology of the bulk spectra of spinons. This time, 
however, the helical liquid of spinons at the surface must 
be coupled to 3D bulk gauge field. Thus the low energy 
properties of topological Mott insulators are dominated 
by the surface helical liquid of spinons coupled to a 3D 
gauge field. This is in contrast to the typical gauge theory 
of spin liquids where the dimensionality of the spinons 
and gauge field are the same. 

In this work, we examine the effect of singular gauge 
field fluctuations on the helical liquid of spinons (Dirac 
fermions with a finite chemical potential). As men- 
tioned above, the gauge field fluctuations coupled to the 
spinons are not confined to the surface but permeate 
the whole system. If the system is thin enough, the 
gauge field fiuctuations associated with the direction nor- 
mal to the surface are not excited and hence the gauge 
field becomes effectively two-dimensional. In which case 
the problem is very analoguous to the standard spinon- 
gauge-boson problem believed to arise in 2D spin liq- 
uid Mott insulators .'22H291 Here the one-loop fermion self- 
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sgn(a;) suggests that the surface 



spinons are not well-defined. On the other hand, if the 
3D nature persists, we find a novel theory in which the 
effective 2D gauge p ropagator is s uppressed by a square 
root: D{v,q) — l/-\/xg^ +iW\/q leading to a marginal 



renormalization of the spinons with the self-energy cor- 
rection being E ^ — ia;ln(l/|w|). The resuhing phase is 
analoguous to, but not the same as, the v — 1/2 com- 
pressible quantum Hall state with unscreened Coulomb 



interaction 
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Notice that the effect of the singular gauge field fluc- 
tuations in 2D regime is known to be non-trivial even in 
the limit of a large number of fermion flavours (N^ : there 
are non-trivial structures in three and higher order loop 
contributions and currently the fate of the theory is not 
entirely knownPSEH Even in this case there may be a fi- 
nite temperature regime where the naive one-loop or the 
leading order large- A^ result may apply before the system 
possibly becomes unstable. In the 3D regime, however, 
the theory is better controlled as we demonstrate in a 
simple scaling analysis and we expect that our results 
obtained here may apply even in the low energy regime. 
A more detailed analysis of higher order loops may be 
necessary to answer this question completely, which is 
beyond the scope of this work. 

The low energy excitations of the topological surface 
spinons and the gauge fields lead to the heat capacity 
scaling as Tln(l/T) in the 3D regime. In contrast to a 
regular 3D strong topological insulator of non-interacting 
electrons, the topological Mott insulator has no charge re- 
sponse at the surface, for example the electrical conduc- 
tivity vanishes and there are no Friedel oscillations asso- 
ciated with a charge impurity. The gapless spinon states 
carry entropy, however, and would give rise to a "metal- 
lic" thermal conductivity. The surface states of topolog- 
ical Mott insulators, just as in regular strong topological 
insulators, mediate a long-range RKKY interaction be- 
tween magnetic impurities deposited on the surface.HSES 
The angle dependence of the generated RKKY interac- 
tion is identical in both cases because the gauge field does 
not break any symmetries of the helical fermion action. 
However, the scaling in terms of the distance between 
magnetic impurities is modified because of the gauge field 
interactions: it is logarithmically suppressed from l/r^ to 
l/(r^ ln(fci?r)). In the 2D regime, the difference is greater 
and the power law may acquire significant corrections.'^ 

The rest of the paper is organized as follows. In sec- 
tion [Hj we define the low energy theory consisting of a 
helical liquid of spinons coupled to a gauge field. The 
effective gauge field dynamics is derived in section |III[ 
where the crossover between the 2D and 3D regimes is 
also discussed. 



In section IV 



the spinon self-energy is 
computed at the one-loop level. In section |VJ a sim- 
ple scaling analysis for the spinon-gauge-field coupling is 
presented. We compute the response functions near Ihp 



in regular topological insulators in section VI This is fol- 
lowed by the computations of 2fci? response functions in 
topological Mott insulators in section |VII[ where the re- 
sulting behavior of the RKKY interactions is discussed. 
We conclude and discuss future directions in section lVHIl 
The appendices show details of some diagrammatic com- 
putations. 




FIG. 1. A schematic representation of the system under con- 
sideration with its low energy excitations. The topologically 
protected gapless spinons (black dots) abide on the shaded 
surface at z = and are coupled to the bulk gauge fluctuta- 
tions (wiggly lines). 



II. MODEL 

The low energy properties of the surface states in a 
strong topological insulator of non-interacting electrons 
can be described by the Hamiltonian^-^ 



H= (fx^p''{x)[-i(T-izx\')-fj]tp{x) 



(1) 



where x = (x, y) labels inplane positions, while z is the 
out-of-plane direction; ■0 is a 2-spinor representing the 
physical spin, cr is the Pauli matrix vector and /x the 
chemical potential relative to the Dirac point. We use 
units where H — vp = 1. This Hamiltonian describes 
massless Dirac fermions at finite chemical potential, the 
latter breaking the emergent Lorentz invariance. 

In the topological Mott insulating phase, the non- 
trivial band topology of the parent topological insulator 
state is transferred to the spinons, which become gapless 
on the surface. In which case, we use Eq. (IT]) as a mini- 
mal model to describe the gapless surface spinon states, 
where ip carries the spin degree of freedom of the original 
electron but not its electric charge. In contrast to the 
strong topological insulator, the surface spinons do not 
propagate freely but are minimally coupled to an emer- 
gent U(l) gauge field Afj,{x,z), where we use x = {t,x), 
T being the Euclidean time. The action of the interacting 
spinons and gauge bosons is given by 

S^Sa+ (fxij^x)i.dr-igAo{x,0)- ^1 

- i(T ■ [z X {\/ ~ igA{x,0))]\^P{x) , (2) 

where Sa — (l/go)^ / d^xdzF^yFf^u is the bulk Maxwell 
action for the 3D gauge field, with field strength tensor 
F^u = d^Ai, — d^A^. The coupling g^ comes from the 
high-energy degrees of freedom associated with the bulk 
fermions. We note that while the gauge bosons exist in 
the bulk, the gapless fermions abide on the surface, which 
we take to be at z = 0, as can be seen in Fig. [T] In this 



work, we focus on a single surface and thus ignore the 
time-reversal partners located on the opposite boundary. 



A. Boundary conditions 

We specify the boundary conditions for the gauge field 
along the ^-direction as follows 



dyAt,ix,z)\ 



^O.L^ 



= 0, n = 0,x,y,z 



(3) 



where Lz is the length of the system in the z-direction. 
This Neumann-type boundary condition corresponds to 
a "free endpoint" type of boundary condition where the 
gauge field can take arbitrary values at the surface as long 
as it does not vary approaching the boundary. We note 
that although the gauge field is confined to the material, 
we do not impose the Dirichlet-type boundary condition 
A^ = on the boundary. This would result in the ab- 
sence of coupling between the surface fermions and the 
gauge bosons, which is not consistent with the micro- 
scopic lattice theory where the boundary fermions are 
always coupled to a lattice-link variable which is a line 
integral of the gauge field. 

By virtue of the boundary condition, we can express 
the gauge field as 



Af^{x,z) = ^ yl„^^^(a;)cos 



TTTlzZ 



(4) 



so that on the surface, we have y4.^(a:,0) = 
S^=o ^m^.mC^)- Substituting this result into Eq. ([2|), we 
see that the surface fermions couple to "standing waves" 
with wave numbers n^ = 0, 1, 2, ... . This additional ex- 
citation channel will play an important role in the low 
energy theory by weakening the gauge fluctuations in 
comparison to a purely two-dimensional theory. 



B. Gauge choice 



We fix the gauge as follows 



(JxAx + OyAy — 



(5) 



In comparison with the standard three-dimensional 
transverse (or Coulomb) gauge, d^Ar^^dyAy + d^Az = 0, 
we do not include the dzA^ term. Notice that our gauge 
fixing condition does not couple A^ to the remaining com- 
ponents. This is convenient because A^ does not directly 
couple to the surface fermions. There might be an indi- 
rect coupling through the bulk fermions that are coupled 
to all the components of the gauge field, but the former 
are gapped and such a coupling will be irrelevant in the 
low energy limit. Thus A^, will not play an important role 
in our discussion and we shall neglect it. We are left with 
two gauge components: the temporal one (Aq) and the 
2D-transverse one (^_l). We shall simply call the latter 
"transverse" in the remainder of the discussion. We now 





FIG. 2. One-loop self-energy corrections to a) the gauge 
boson and b) fermion. The solid lines denote the free fermions 
while the curly lines represent the transverse gauge bosons. 



turn to the renormalization of the gauge field by the low 
energy particle-hole excitations of surface spinous. 



III. GAUGE FIELD DYNAMICS 

We first write the free-fermion action in frequency- 
momentum space: 5/ = / tppGp^ipp, where we have 
used the energy- momentum 3- vector p = ipo,Px,Py) and 
J = J 12^)3 ■ (It should be clear from the context when 
instead we mean p ~ |p| .) The inverse propagator matrix 
is 



Gp ^ = iw -f- ^ + py(7x - PxO-y 



iuj + n py + ipx 
Py - ipx iuj + ^1 



(6) 



We can conveniently express the matrix propagator aa^ 
Gp = J2Ps{0p)Gsip), (7) 



s=± 



where 



Ps{0p) = l[l + SZ-{pXa)] 

1 / 1 -sie-''^p 



and 



2 Xsie^'^p 1 



G-\p)=iL^-^, 



(8) 



(9) 



with 9p being the angle of p relative to the a;-axis and 
^s,p — sp~^. We note that the propagator G_ — [iuj—p— 
/i)"^ remains finite for oj = and p = /x; it corresponds 
to fermions in the lower Dirac cone. Lying far from the 
Fermi surface, they do not contribute to the low energy 
properties. We further note that if the chemical potential 
is tuned to zero, one is left with Dirac fermions and the 
above decomposition, Eq. ([t]), is not useful; instead, one 
can perform the calculations by exploiting the Lorentz 
invariance of the action. 

We evaluate the renormalization of the gauge field 
components Aq and A^^ due to the gapless surface 
spinous by calculating the bosonic self-energy at one loop 



(Fig. [2]ja)). It will be useful to consider a general polar- 
ization function with incoming and outgoing interaction 
vertices 7a and 7b, respectively. The vertices are matrices 
in spin space and can in general dependent on momen- 
tum. The corresponding polarization function reads 

Uab{q) = f tlijaGp%Gp+g] , (10) 

Jp 

= 9a9bY. I Gs{p)GAp + q)rJ , (11) 

.. ,,' = -!- •'P 



s.s' — ± 

where ga,b are the coupling strengths and 



(12) 



is a form factor arising from the combination of the angle 
dependence of the fermionic propagators and the vertices. 
It will in general be different for various polarization func- 
tions, as we shall see. It has also been normalized so as 
to be independent of the coupling strengths. 

We consider first the boson self-energy for the trans- 
verse gauge component, A_\_. One can simply see that the 
vertices are momentum dependent: 7a(b) = ±5<? • c- The 
expression for the polarization function becomes 



nj.j.(<z) = g2 ^ Gsip)GAp + q)l 



l±± 



-.±-'p 



where 



m\ = -1 



I p cos 26 + qcosi 



\P- 



(13) 



(14) 



and 9 — p ■ q is the angular integration variable. In the 
limit where the external frequency-momentum vector is 
zero, we find Tl±±{qp, = 0) = —g^A, where A is an arbi- 
trary ultraviolet (UV) cutoff. We regularize this expres- 
sion so that the transverse compo nent remains massless, 
as is required by gauge invariance^SESl. For finite g^, we 
obtain 



n. 



xq +7- 



q 



(15) 



where x = coS^/a* and 7 = ci.g^/i; the ct are positive real 
numbers. This result is valid for small momenta q <^ kp 
and in the static limit |i^| <^ q. 

Next, we turn to the polarization function for the tem- 
poral component Aq. The vertices are simply 7a(f,) = gl 
because Aq couples to the spinon density; as a result the 
form factor becomes 



2/0, 



1 



,p + qcosO 



\P- 



(16) 



We find noo(g^ = 0) ex —gfJ-, i.e. the Fermi surface 
of spinous induces a mass for the temporal gauge com- 
ponent. We note that this mass does not correspond 
to a UV cutoff, and cannot be removed by the above 
regularization. We also note that the mixed polarization 



functions vanish exactly no_L = IIj^o = 0. At finite chem- 
ical potential, we can thus neglect the massive (screened) 
temporal gauge component in comparison with the trans- 
verse one, which mediates a long-range interaction be- 
tween the spinous. 

We now write down the effective gaussian action (or 
RPA action) for the transverse gauge field by retain- 
ing the dominant contributions from the bulk Maxwell 
Lagrangian and from the self-energy due to the helical 
fermions: 






+ xq +7- 



where Xz — L^/g"^- The gauge propagator is then 



T^n A<l) = 



X: 



I 7rn^ \ 



xq 



■7 



M 



(17) 



(18) 



The key difference from the finite chemical potential 
QED3 is the appearance of n^ encoding the possible fluc- 
tuations of the gauge field in the out-of-plane direction. 
We note that in the effective action we have only re- 
tained terms with identical incoming and outgoing stand- 
ing wave numbers as these processes dominate in the low- 
energy limit. 



A. 2D-3D crossover and effective gauge field 
propagator 

When we evaluate amplitudes for quantum processes 
involving virtual gauge field fluctuations such as the 
fermion self-energy, the fermion-gauge-boson vertex, etc, 
we will need to sum over all possible bulk standing waves 
of the transverse gauge-field. Since the fermions have no 
dependence on the standing wave number we can simply 
sum the gauge field propagator over all n^ to obtain an 
effective 2D propagator: 



D{q)^Y.^„M 



(19) 



In the limit of infinite L^, the sum becomes an integral; 
it can be evaluated simply by rescaling the integration 
variable: 



D{q) 



h 

71- JO 
rl/2 

goLz 



dqz 



1 



(20) 



Xzq^z+T^^Q)-' ' 
V^^) , (21) 



where 'Do{q)~^ = xq^ + iWl/q is the standard, inverse 
gauge field propagator in the absence of bulk fluctuations. 
Note that the effective 2D gauge field propagator is less 



singular as g^ — >■ than the purely 2D propagator 'Do{q). 
This is to be expected as the flucutations have access to 
more phase space. 

For finite Lz, the discrete sum can be done ex- 
actly: D{q) = h{glLzVo{q)-^), with h{x) = (1 + 
y/xcoi\iy/x)/2x. The function h interpolates between 
the purely two- and three-dimensional regimes 



h{x) 



1 


j2/x 


, a;< 1, 


2D regime 


2 


l/^/x 


, x> 1, 


3D regime 



(22) 



Let us now examine the crossover between these two 
regimes. First, we consider the zero temperature case. 
The condition a; ^ 1 for the occurence of the 2D regime 
corresponds to 



2 ^ 1 

xq + 7— « ^7 



(23) 



In the limit of infinite inplane area, one can always satisfy 
this inequality provided one takes jg^j sufficiently small, 
i.e. for small enough \q^\ the fluctuations behave in a 
purely 2D fashion, as is expected since the inplane di- 
mensions arc infinite while the out-of-plane one is finite. 
For a finite system, it would be more meaningful to put a 
lower bound on the inplane momenta, qi > 2Tr/L, where 
L characterizes the size of the inplane dimensions. We 
can further use the scaling relation ly ^ {x/l)l^ for the 
Landau damped gauge field. Combining everything we 
obtain L^ <^ fiL'^/g'^gQ. 

At finite temperature, one can use v ^ T together with 
V ^ (x/t)?^ to obtain the following inequality for the oc- 
curence of the 2D regime for the gauge field fluctuations 



i^< 



1 



9 QofJ' 



(24) 



As expected, as one increases the temperature, the sam- 
ple would have to be thinner in order for the gauge fluc- 
tuations to be of two-dimensional nature. 

To summarize, the gauge fluctuations can be of either 
2D nature (strong) or 3D nature (weak) depending on the 
system parameters L^, T, /x. The effective 2D propagator 
corresponding to each of these regimes is 



D{q) 



xq'^ + i\v\/q 

dp 

\/xf~+iH/q 



2D regime 



3D regime 



(25) 



1 /2 

with do = .90^2 /2. As can be noted, in the 2D regime 
the low energy theory for the topological Mott insulator 
becomes analoguous to the standard spinon-gauge-fleld 
problem, which, as mentionned in the introduction, was 
extensively studied and still lacks a satisfactory under- 
standing. On the other hand, the 3D regime has weaker 
gauge fluctuations and this will make the theory more 
tractable. In the remainder of the work, we shall focus 
mainly on the 3D regime. 



B. Heat capacity from gauge fluctuations 

Using the gaussian action for the transverse component 
of the gauge fleld, the low temperature heat capacity can 
be shown to scale like C ^ Tlnl/T in the 3D regime 
while it scales like C ^ T^/^ in the 2D regime.'*^ In the 
absence of the topological surface states, the main contri- 
bution to the heat capacity would come from the gapless 
gauge bosons, yielding a much smaller contribution scal- 
ing like T^. The phonons would also contribute the same 
power law. 



IV. SPINON SELF-ENERGY 

We evaluate the one-loop spinon self-energy due to 
gauge field fiuctuations. As is well known, the main effect 
of the over-damped gauge field is the appearance of non- 
analytic frequency dependence of the self-energy. The an- 
gular dependence of the fcrmionic propagator, however, 
is not changed as the coupling to the gauge field does not 
break the symmetries of the original free fermion system 
(this is discussed in detail later in this section) . Formally, 
the one-loop self-energy matrix reads (Fig. l2jb)) 



S(p)=ff2 / D(q){q-(T)G{p + q){q-(T) 



(26) 



where the fermion propagators are given by Eq. ([7|. We 
quickly note that G^^ = G^ so that E^^ = E^^. The off- 
diagonal elements are also simply related: 'S^i{oj,p)* = 
Ei-|-(— aj,p). For u '^ and p '^ fi, we find (see appendix 

IaI 



S(P) 



E^-t-(w), 



(27) 



with the same frequency dependence for all components 
of the self-energy matrix: 



^n\ 



'A2Sgn(w)|wp/3, 
^Ag^ln^, 



2D 
3D . 



(28) 



^2,3 are positive constants: A2 — C2/i^''^ and A3 = 
c^gfi^'^goLj , where the Ci are positive order one num- 
bers. We see that in the 2D limit, where the gauge fluc- 
tuations are the strongest, we obtain the standard scal- 
ing ^ I a; p/^. 1221331 In the 3D limit, we instead obtain the 
marginal correction ^ wln|a;|, which is only logarithmi- 
cally larger than the bare dynamical term. 

The Dyson equation with one- loop self-energy, Q^^ = 
G^^ — E, yields the dressed fcrmionic propagator 



^p-E^''(^p)^«(p) 



s=± 



where the dressed function Q^ reads 

G+^ = iuj - E^^(a;) - S.p 



(29) 



(30) 



while Q- = G- remains unchanged at leading order. As 
expected the angle-dependence of the dressed propagator 
remains the same as for the bare one, while the dynamical 
term is renormalized. 

The above result is consistent with symmetries. In- 
deed, the free fermion Hamiltonian, Eq. ([l|, is invariant 
under continuous rotations of the momentum and spin 
of the fermions by the same angle. From a more general 
field theory perspective, this spin-momentum rotation is 
a Lorentz transformation, under which the spinor natu- 
rally transforms under a two-dimensional representation 
of the symmetry group. Consider such a clockwise rota- 
tion by an angle a: 






(31) 

(32) 



where Ra — e*""^ , t'' denoting Pauli matrices acting in 
coordinate space; e^'i"' = diag(e*"/^,e~*"/^). The ac- 
tion depends only on the relative angle between the spin 
and momentum of fermions, via the term z • (cr x V), and 
thus remains invariant under a joint rotation of the spin 
and momentum by the same angle a. The same holds 
true in the presence of the gauge field, the components 
of the latter transforming in the same way as the coor- 
dinates. The symmetry requires (e*^*^ ippip^e^^^"^ ) — 

{'4'po,RapVL R „)■ The left hand side can be shown to 
give: 



' (V'pV'p 



^■ft 



'Sn 



e-^'^Glt Su 



(33) 



which agrees with the angle dependence obtained for the 
fermionic propagators as can be seen by letting 6p — >■ 
Op- am Eq. pO]). 



where the new 2-spinor, /''" = (/+, /_), is related to the 
original one by a SU(2) transformation 






1 

71 



1 



1 



(36) 



We recognize that the diagonal elements of the new ac- 
tion, Eq. (35), correspond to the inverse propagation am- 



plitudes for fermions in the upper and lower Dirac cones, 
G^^. At finite chemical potential (and temperature less 
than /i), we can omit the /_ fermions as they are gapped. 
The entire action expanded near {kp,0) reads 



(fk fk{iv - v^k^ - lykl)fk 



+ g f dq^d'qd'k A,^ {q)h+,h , (37) 



where, at the bare level, Vx — ^ and ly — l/2fi. We have 
omitted the subscript -I- from the fermion fields. Ak^ (k) 
is the cc-component of the gauge field, i.e. the one that 
couples the most with the fermions at {kp, 0). Note that 
we are using the continuous limit for bulk excitations of 
the gauge field. 



We first rescale the frequency and momenta: 



(38) 
(39) 
(40) 
(41) 



We note that not a single dynamical exponent z will keep 
the entire action invariant because the fermions have a 



ly' 


^biy , 


K 


= bk^ , 


K 


= b'/'ky , 


fci 


= b^/'k, . 



dynamical exponent 2/ 
the bosons have Zf, = 3. 



2 (with respect to ky) while 



V. SCALING ANALYSIS 



If we make the choice z = 3, we find 



In this section, we provide a scaling analysis of the 
spinon-gauge-boson system in the 3D regime in order to 
analyze the effect of the gauge fluctuations. We start 
with the free fermionic action which can be written as 



[ly] - 1/3 , 

[5] = , 
with all other couplings being marginal. 



(42) 
(43) 



Sf = 



ipl[iu) + ^i + Pya"" -Px<y^]ipp 



(34) 



We carry our analysis on a patch of the Fermi surface 
near (kp, 0); we express the momenta as deviations from 
this point: p = (kp -f- k^, ky), where \kx\, \ky\ <Si kp. The 
analysis is further simplified in a basis that diagonalizes 
the free-fermion action: 



ft 



iuj~^p 
iiu 







fp , (35) 



If instead we use z = 2 all couplings in the fermionic 
action naturally remain marginal. One then has a choice 
for the scaling exponent of the gauge field by either re- 
quiring that the ky term or the Landau damping term 
remain invariant. The first choice would be the natural 
one had we started with the minimal action for the gauge 
field, i.e. the same action as above except with the omis- 
sion of the Landau damping term (which is generated 
in the one- loop bosonic self-energy). Thus keeping the 
ky term invariant, we obtain the following engineering 



dimensions 



[ly] = , 

[7] = 1/2 

[9] - , 



(44) 
(45) 
(46) 



related to the physically relevant RKKY interactions on 
the surface of the TMI. 



with all other couplings being marginal. Notice that both 
scaling choices {z — 3 and z — 2) make the coupling to 
the gauge field marginal, which is consistent with the 
one-loop self-energy correction acquiring only a logarith- 
mic correction. With these naive scaling results in mind, 
we now study the RG flow at the one-loop level. Fol- 
lowing Mross et al.'^', we integrate out the modes whose 
momentum lies in the shell A/6^/^ < \ky\ < A, A ^ kp 
being the UV cutoff. To order g'^, the fermionic dynam- 
ical term gets modified by the self-energy diagram to 



iv{l + cg'^liib) -^ ivlf^ 



(47) 



where c — f /(tt^x^ -2;). Note that this leading correction 
is independent of the Landau damping of the gauge boson 
so that it is the same whether one includes the Landau 
damping term or not. The correction to the the vertex 
vanishes because at zero external frequency the integrand 
of the kx integral has its two poles in the same half-plane. 
The dynamical term of the gauge field remains invariant 
because the integration of high-|fcy| modes leads to the 
generation of irrelevant operators. 

The self-energy correction can be incorporated in a 
modified frequency scaling 

v' = vh^+^a^ . (48) 

We use z — Z and keep the same scaling for k^ , ky and 
kz as above. Using 6 = e', the differential RG equations 
read 



dly 
111 

d'j 

u 

dl 



3 ' 



-eg 



(49) 
(50) 
(51) 



We see that the coupling g has become marginally irrel- 
evant. If instead we use the z = 2 scaling, we obtain 



^7 

dg^ 
dl 



7 



cg^i 



-eg 



(52) 
(53) 



with all other parameters remaining invariant. Thus, in 
both cases, the coupling to the gauge field is marginally 
irrelevant and the theory seems well controlled once the 
dressed fermion propagator with the one-loop self-energy 
is used. 

Now that we have a better understanding of the low- 
energy properties of the coupled system, we turn to the 
investigation of the 2kp polarization functions, which are 



VI. BARE 2kF CORRELATION FUNCTIONS 

FOR THE SURFACE STATES OF A 

TOPOLOGICAL INSULATOR 



We give the results for the bare, static, spin-spin po- 
larization functions near 2k p, whose fourier transform 
yields the spinon mediated RKKY interactions between 
surface impurities. We note that if we neglect the gauge 
fluctuations, the polarization functions are naturally the 
same as for a strong topological insulator, as given bySl 
for instance. More precisely, we will evaluate Eq. ( [Tl] ), 
with vertices tr*, i — x,y,z. We evaluate the form fac- 



tors, deflned in Eq. (12), in the limit appropriate for the 



external momentum being near a nesting vector of the 
Fermi surface, q ~ 2kF. Retaining only s — s' = + and 
defining fij = /^t^, with i,j S {x,y,z}, to leading or- 
der we find that these form factors do not depend on the 
internal angle but only on the external one, 6q. We can 
thus write 



n„(o,q) = /,j(0,)n(o,q). 



where 



n(o,q) 



G+{p)G+ip + q) , 



(54) 



(55) 



and the angle dependence reads 

Jij 



cos"«^ isin20q —icostq 
^sm29q sin^ 9q —iaindq 
i cos 0„ i sin 9„ 1 



(56) 



The non-analytic dependence near 2kp obeys the stan- 
dard scaling in two dimensions with exponent 1/2: 

where 6q ^ q — 2kp. This result is valid for \6q\/kp <C 1. 
One can perform a Fourier transform to real space 



X^J{r)^ Jd^q^,,{0,q)e'1■'■ 



{58) 



This leads to a RKKY interaction term between two im- 
purity spins 5^1(2) located at rx(2) of the following form: 

i^RKKY - Jl(0^1^2 + Mr){Si ■ f){S2 ■ f) 

+ J2{r)f ■ {Si X S2) , (59) 





FIG. 4. 2kF dressed polarization function. The double line 
represents the dressed fermionic propagator. 



FIG. 3. 2kF vertex correction. The black dot represents 
any spin vertex and we use double lines to represent dressed 
fermionic propagators. 



where r = ri — r2- For a sufficiently large separation 
between the impurity spins, 2kpr 3> 1, we have 



sin 2kpr 
Ji\n oc ^ , 

, , cos2kpr 
J2\r) oc ^ 



(60) 



We have dropped subleading terms decaying as \/r^. 
These induced RKKY interactions on the surface of a 
strong topological insulator were also examined bjB^ISI 
using a different approach. 



VII. DRESSED 2kF CORRELATION 

FUNCTIONS FOR THE TOPOLOGICAL MOTT 

INSULATOR 

In this section we will determine the effects of the gauge 
fluctuations on the RKKY interactions on the surface of 
a topological Mott insulator and we will show that they 
differ from the regular topological insulator. 



A. Vertex corrections 

We first examine the effect of low-energy gauge bosons 
on the 2kp vertices. For a general vertex 7a, the correc- 
tion we need to evaluate at one loop is (Fig. [S]) : 



Jk 

xg{p-k)j^{-k)D{k) . (61) 

We use the dressed fermionic propagators. We set the 
external frequency qo to zero. The singular behavior is 
expected to occur when the incoming fermion is on the 
Fermi surface, i.e. p ~ fci?, and is backscattered by the 
external boson so that q ^ — 2p. The internal transverse 
gauge fluctuation has small A:^. As before, the vertex 
involving the latter is simply j±{k) = gk ■ cr. Using the 
decomposition for the fermionic propagators we can write 



it more simply as 



5^a 



X fGs{p~k)gAp + q-k)D{k)F^' 



(62) 



where -Flf* is a matrix form factor: 



F:' = {k ■ <T)P,,{e^+^_k)%Ps{Op-k){-k ■ a) . (63) 

Note that the form factor is a polynomial of cosines and 
sines with constant coefficients. Although it can never 
diverge, it can potentially suppress the amplitude. In 
appendix [B] we show that the one-loop vertex correction 
with the form factor matrix set to a constant is not sin- 
gular in the infrared (IR) when go = 0,(7 = 2kp,pQ = 
0,p — — g/2. Thus the vertex corrections S'ji, i — x,y, z, 
will not alter the non-analycity near 2kp. This results 
from the suppression of the effective gauge propagator in 
the 3D regime and not from the smearing of quasiparti- 
cles. Indeed, the vertex correction computed with bare 
fermionic propagators is also IR-convergent. 



B. 2kF spin-spin correlators and RKKY interaction 

The gauge fluctuations will affect the 2kp correlation 
functions due to the smearing of the quasiparticles. The 
result will be a suppression of the non-analyticities. The 
vertex corrections, lacking singularities, will not compen- 
sate this effect. The angle dependence they carry, see 
Eq. ( B3 ) , will not result in new angle dependence for the 



2kp correlators as we demonstrate in appendix [C| This 
is in agreement with the fact that the gauge field does 
not break any symmetries present in the free fermionic 
action, hence the spin-spin correlation functions should 
retain the same angle dependence after the inclusion of 
gauge fluctuations. 

The renormalized polarization functions in the TMI 
read (Fig. ^ 



n-j(g) = / tri^Gp^jCp+q] , 



(64) 



where we have used the dressed fermionic propagators. 
As we argued above, the angle dependence of the later 



is the same as in the free theory, which results in the 
same form factors /y, Eq. (56). Thus, lllj{0,q) = 
fij{9q)W{0,q). Next, we needro evaluate the two-point 
correlator with form factor set to one: 



Jn 



oc — /i ' / duj xs 

10 



1/2 / doj' 



^^Sg + i\3Uj\n^ 



(65) 



where we have evaluated the momentum integrals and 
dropped the bare dynamical term. The remaining inte- 
gral cannot be done analytically, however one can extract 
the leading singular momentum dependence: 



nsi„g(0,g) oc 



1^. 



11/2 



'-ik 



(66) 



This can be seen to lead to a logarithmic modification 
of the real space scaling found at the bare level. Indeed, 
the singularly at 2kp is logarithmically weakened and 
this will lead to a real space scaling that decays faster. 
Instead of the bare scaling l/r^, we get l/{r'^ \nkpr). 

In the 2D regime, we find that at one-loop the spin 
vertices are logarithmically enhanced^ Sj ex lnyu/|5q|, 
where Sj is the 2/cp vertex without form factor, see 
Eq. (B2|. In this case, the vertices are singular and will 



contribute to the spin-spin correlation functions. One can 
resum the logarithms arising from all the ladder diagrams 
to obtain a power law:^^ The exact exponent, however, 
depends on the pre-factor of the logarithm and its value 
has been known only in some limits. I^S This would also 
modify the exponent of the RKKY interaction. For the 
density-density correlation, the form factor can be shown 
to render the vertex IR convergent. 



VIII. CONCLUSION AND DISCUSSION 



system and temperature. 

In the 3D regime, where the fluctuations in the bulk 
are strong, the effective 2D gauge field propagator can be 
obtained after summing over all standing-wave modes. 
It turns out that this "holographic" 2D propagator 
D{v,q) ~ l/'v/x'Z^ + iWMq is less singular than its 2D 
counterpart Do{i^,q) = l/(x9^ +71^1/9)- As a result, 
the spinon self-energy correction due to the gauge field 
is only logarithmically singular, S ~ — ia;ln(l/|ci;|). It 
was shown that this leads to a more controlled theory 
for the spinon-gauge-field system, analogous to the case 
of the 1/ = 1/2 compressible state in the quantum Hall 
regime. In the 2D regime, however, the one-loop self- 



energy E 



'l\UJ 



2/3 



sgn(a;) suggests that the surface 



spinous are not well-defined. Moreover, according to re- 
cent developments in the theory of 2D spinon-gauge-field 
system, the ultimate fate of the theory is not known even 
in the large- iV limit .I^SiSllxhe same problem exists in the 
2D regime in our case and we do not attempt address 
this question in our current work. We emphasize, how- 
ever, that the 3D regime is better controlled as we argued 
in the main part of the paper. 

It is shown that the 3D regime leads to Tln(l/T) spe- 
cific heat behavior and the RKKY interaction between 
magnetic impurities on the surface would be suppressed 
by a logarithmic factor. Perhaps the most fundamental 
difference between the surface state of the usual topo- 
logical insulators and topological Mott insulators is that 
both the Friedel oscillations of charge density and the 
RKKY interaction exist on the surface of the topological 
insulator while only the latter (with logarithmic correc- 
tions) arises in the topological Mott insulator. It is also 
pointed out that the "metallic" thermal transport should 
exist at the surface of the topological Mott insulator due 
to the entropy carried by the surface spinous while there 
is no charge transport. 



We considered the effect of gauge field fluctuations in 
topological Mott insulators, which can be regarded as 
spin liquids with the bulk spinous possessing a topolog- 
ically non-trivial band structure. The manifestation of 
the non-trivial topology of the bulk-spinon spectra is the 
presence of the gapless surface-spinon state, a helical liq- 
uid of spinous. 21 The low energy properties of this sys- 
tem, therefore, are dominated by the surface spinon state 
coupled to a three-dimensional emergent U(l) gauge field 
in the bulk. 

Notice that this is a unique situation: in the usual spin 
liquid state derived from spin models, the dimensio nality 
of the gauge field is the same as that of the spinons.'^^Ml 
In the finite thickness geometry, the three-dimensional 
nature of the gauge field is represented by the "standing 
waves" associated with the degrees of freedom perpen- 
dicular to the surface. It is shown that there would be a 
crossover between the 2D and 3D regimes in the behav- 
ior of the gauge field, depending on the thickness of the 



One important topic that we have not discussed so far 
is the presence of the term in the effective gauge field 
action.BSMI Because of the topological band structure of 
the spinous, the so-called axion term, ^E ■ B, would 
arise in the effective action of the emergent gauge field, 
where the action is 27r periodic in 9. This means that 
the monopoles in the compact U(l) gauge theory would 
possess both "electric" and "magnetic" charges of the 
emergent gauge field .S3 The condensation of these dyons 
will drive the confinement-deconfinement transition. In 
the deconfined phase of the topological Mott insulator, 
these dyons are gapped and do not play an important 
role in the low energy dynamics. Thus the low energy 
behaviors discussed in our work should not be changed. 
In the confined phase, however, the dyon condensation 
may lead to novel symmetry-breaking ground state (of 
usual electrons) due to mixed quantum numbers carried 
by them, which would be an interesting topic for future 
study. 
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Appendix A: Fermion self-energy 



We evaluate the fermionic self-energy, Eq. ( 26 ) : 



S«.g^ D{q)G+{p + q)iy, 



e-'"" 



e'"" 



1 



-le 



-lUp-Yq 



1 



= g' D{q)G+{p + q) 



~ie 



e-"-^" 
e^^" 

1 

-iep^q+i29q 



(Al) 



11 f}''^ "P -\- q IjiiUq 



1 



(A2) 



To leading order 0p+q ~ 9p since p ^ kp and q ^ 0. Also 
the relative angle between the p and q, 9p — dq, is near 
±7r/2, so that the scattered fermion stays near the Fermi 
surface. This leads to 



9 



1 —ie 

gi8p 2 



D{q)G+{p + q). (A3) 



We can see that the off-diagonal elements have the same 
angle dependence as the bare inverse fermion propagator, 
which leads to no new angle dependence in the dressed 
propagator. Although we have used approximations to 
determine the angle dependence of the self-energy, it 
should be emphasized that this holds exactly, as we have 
verified explicitly. It also follows from symmetry consid- 
erations (see section IV). 

-iAawln/x/jwI. We 
Given the ex- 



The remaining integral is given by 
show the main steps of the calculation, 
ternal fermionic momentum vector p, we decompose the 
bosonic internal momentum q along directions parallel 
and perpendicular to it: g|| = q ■ p and q± = q ■ {z x p). 

2 

For p ^ 11 and q <t^ ^,we have £,p+q ~ 9|| + f^ • ^^ drop gy 

in the bosonic propagator: D{ q) — do/y^xQ^ +iW/q±\- 
The remaining integral in Eq. (|A3|) is given by 



dv dq\\ dqj 



do 



1 



2-K 27r 27r 



v/7S+X9i*(^ + ^)-(9|| + fc) 



_._do_ 
cx —iio In - 



dv , M A^ 

-sgn(c. + .)ln^ 



(A4) 



where we have performed first the q\\ integral by con- 
tour integration (using a regulator, for instance), then 
extracted the leading logarithm from the qj^ integral. 



We show the calculation of the vertex corrections. We 
illustrate the analysis for 5^^. At leading order we have 

^7. = -g' J g+{p-k)g+{p+q-k)D{k)F++ , (Bi) 

where the form factor is 



F++ = (fc • CT)P+{ep+q^k)'y'P+{ep-k){~k ■ <t) 

__l/0 e-'^>'\ f 1 -ie-''^-p\ fl 
"* 4 U^^^- / Ue*«-p 1 / U -1 







1 f 1 ie^" 

2 Ue^^'J -1 



In going from the first to the second equality, we have 
dropped k as it will give rise to subleading corrections. 
We are also setting q — —2p. In the last equality we 
have used Op ~ 0k ~ ±7r/2, i.e. the fermion couples most 
strongly to a gauge fiuctuation whose momentum is per- 
pendicular to its own. We see that to leading order the 
form factor matrix is independent of the internal mo- 
mentum, hence there can be no suppression for this ver- 
tex. It remains to calculate the remaining integral with 
form factor matrix omitted. For q = 2kp, the vertex 
will receive its largest contribution when the fermions lie 
near the Fermi surface, so that the fermionic momenta 
are near ±q/2, which are antipodal points of the Fermi 
surface. We expand the fermionic dispersion relations 
about these two points: £,±3.-k — i^H + k'j_/2fi, where 
fc|| = —kpq ■ k and fcj_ is the momentum in the transverse 
direction. We evaluate the vertex correction using Po = 
and p — — q/2 = —kpq. We drop the bare fermionic dy- 
namical term and retain only the self-energy correction, 
which reads S = —iX3i'lnfi/\i'\. The details of the cal- 
culations are 



Sj ; 



V j Q+{p-k)g+{p + q-k)D{k), 
Jk 



(B2) 



= -9 



1 1 

;, -S + fc|| - kl/2fi -E - /s|i - fc2_/2/i 
do 



X 



Vxkl+lW/k^l 



dvdk I 



do 



iT, 



Vxk'i + iW/k^\ m^ + {ki/2,,y 



By performing the kj_ integral first and then the fre- 
quency integral it can be seen that there is no IR di- 
vergence. This is contrast to the half-filled fractional 
quantum hall scenario, where the propagator is D~^ — 
x\k±\ +j\h'/k±\. In that case one obtains, using dressed 
fermions, a weak IR divergence ~ (In | In Aift|)^. 

We deduce that the all the one-loop vertex corrections 
are IR convergent near 2kp. We conclude by giving the 
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explicit angle dependence for the three spin-vertex cor- 
rections: 

^"f^^l[^k '-1 j-^^' 
1 /-ie*"" 1 \ 

1 /-ie-*^" e-*20, 



^^- = l[\ ;-..,j^7, (B3) 

where (57 > is a constant. Note that S'y± corresponds 
to the bare vertex cr^ = {a^ ± io'^)/2. 



Appendix C: Angle dependence of 2kF polarization 
functions with dressed vertices 



We explicitly verify that the angle dependence ob- 
tained for the vertex corrections in the previous appendix 
does not lead to a different angle dependence in the po- 
larization functions near 2kp when compared with the 
free-fermion theory, Eq. (56). We shall only verify this 
for the n^2 polarization function, the calculation being 



very similar for the other polarization functions. Includ- 
ing the vertex corrections, the polarization function reads 



nL(9)= h''^[izip,q)Gp7z{~p,-q)Gp+q] , (Ci) 



where jz{p, <l) = <^^ + Sjzip, q)- We evaluate the polariza- 
tion function at zero external frequency and for the ex- 
ternal momentum equal to a nesting vector of the Fermi 
surface: qo = and q = 2kp. This amplitude will re- 
ceive its largest contribution when the particle and hole 
lie near the Fermi surface, so that the fermionic momenta 
are near ±q/2. In the previous section, we have shown 
that the one-loop vertex correction is not singular near 
2kp, hence, to leading order, we can evaluate it for po = 
and p — — g/2 = —kpq, as given in Eq. (B3|. We are 
thus left with 

KMq)= lQ+{p)Q+{p + q) 
jp 

•Ki,[Uq)P+{Op)%{~q)P+{ep+,)] , (C2) 

where we have again retained only the term with s = s' = 
+ and have defined 7z(g) = 7z(Po = 0,p — — q/2;qo — 
0, q = 2kFq)- To obtain the angle dependence we eval- 
uate the trace in the last equation. To leading order we 
can take 6p = d-q/2 smd 0p+q ~ ^g/2- In which case, 
the trace becomes independent of the integration vari- 
able and one can evaluate it to find that it is a constant 
independent of 0q, just as in the bare case. This is con- 
sistent with the fact that the gauge field does not break 
any symmetry present in the free fermion action. 
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